Acting Optimally in Partially Observable Stochastic Domains
نویسندگان
چکیده
In this paper, we describe the partially observable Markov decision process (pomdp) approach to nding optimal or near-optimal control strategies for partially observable stochastic environments, given a complete model of the environment. The pomdp approach was originally developed in the operations research community and provides a formal basis for planning problems that have been of interest to the AI community. We found the existing algorithms for computing optimal control strategies to be highly computationally ine cient and have developed a new algorithm that is empirically more e cient. We sketch this algorithm and present preliminary results on several small problems that illustrate important properties of the pomdp approach.
منابع مشابه
Bibliography for Tutorial on Statistical Approaches to Spoken Dialogue Systems
[1] AR Cassandra, LP Kaelbling, and ML Littman. Acting optimally in partially observable stochastic domains. In Proc Conf on Artificial Intelligence, (AAAI), Seattle, 1994. [2] F Jensen. Bayesian networks and decision graphs. Springer Verlag, 2001. [3] L Kaelbling, ML Littman, and AR Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101:99–134, ...
متن کاملLearning and Planning in Multiagent POMDPs Using Finite-State Models of Other Agents
My thesis work provides a new framework for planning in multiagent, stochastic, partially observable domains with little knowledge about other agents. The relevance of the contribution lays in the variety of practical applications this approach can help tackling, given the very generic assumptions about the environment and the other agents. In order to cope with this level of generality, Bayesi...
متن کاملDynamic Decision Making in Stochastic Partially Observable Medical Domains: Ischemic Heart Disease Example
The focus of this paper is the framework of partially observable Markov decision processes (POMDPs) and its role in modeling and solving complex dynamic decision problems in stochastic and partially observable medical domains. The paper summarizes some of the basic features of the POMDP framework and explores its potential in solving the problem of the management of the patient with chronic isc...
متن کاملDecayed Markov Chain Monte Carlo for Interactive POMDPs
To act optimally in a partially observable, stochastic and multi-agent environment, an autonomous agent needs to maintain a belief of the world at any given time. An extension of partially observable Markov decision processes (POMDPs), called interactive POMDPs (I-POMDPs), provides a principled framework for planning and acting in such settings. I-POMDP augments the POMDP beliefs by including m...
متن کاملPC-SHOP: A Probabilistic-Conditional Hierarchical Task Planner
SOMMARIO/ABSTRACT In this paper we report on the extension of the classical HTN planner SHOP to plan in partially observable domains with uncertainty. Our algorithm PC-SHOP uses belief states to handle situations involving incomplete and uncertain information about the state of the world. Sensing and acting are integrated in the primitive actions through the use of a stochastic model. PC-SHOP i...
متن کامل